Two-person Zero-sum Differential Game Research Articles

A differential game control problem with state constraints

<p style='text-indent:20px;'>We study the Hamilton-Jacobi (HJ) approach for a two-person zero-sum differential game with state constraints and where controls of the two players are coupled within the dynamics, the state constraints and the cost functions. It is known for such problems that the value function may be discontinuous and its characterization by means of an HJ equation requires some controllability assumptions involving the dynamics and the set of state constraints. In this work, we characterize this value function through an auxiliary differential game free of state constraints. Furthermore, we establish a link between the optimal strategies of the constrained problem and those of the auxiliary problem and we present a general approach allowing to construct approximated optimal feedbacks to the constrained differential game for both players. Finally, an aircraft landing problem in the presence of wind disturbances is given as an illustrative numerical example.</p>

A Dimension-reduction method for the finite-horizon spacecraft pursuit-evasion game

<p style='text-indent:20px;'>The finite-horizon two-person zero-sum differential game is a significant technology to solve the finite-horizon spacecraft pursuit-evasion game (SPE game). Considering that the saddle point solution of the differential game usually results in solving a high-dimensional (24 dimensional in this paper) two-point boundary value problem (TPBVP) that is challengeable, a dimension-reduction method is proposed in this paper to simplify the solution of the 24-dimensional TPBVP related to the finite-horizon SPE game and to improve the efficiency of the saddle point solution. In this method, firstly, the 24-dimensional TPBVP can be simplified to a 12-dimensional TPBVP by using the linearization of the spacecraft dynamics; then the adjoint variables associated with the relative state variables between the pursuer and evader can be expressed in the form of state transition; after that, based on the necessary conditions for the saddle point solution and the adjoint variables in the form of state transition, the 12-dimensional TPBVP can be transformed into the solving of 6-dimensional nonlinear equations; finally, a hybrid numerical algorithm is developed to solve the nonlinear equations so as to obtain the saddle point solution. The simulation results show that the proposed method can effectively obtain the saddle point solution and is robust to the interception time, the orbital altitude and the initial relative states between the pursuer and evader.</p>

Generic uniqueness of saddle point for two-person zero-sum differential games

Abstract The generic uniqueness of saddle point for two-person zero-sum differential games, within the class of open-loop, against the perturbation of the right-hand side function of the control system is investigated. By employing set-valued mapping theory, it is proved that the majority of the two-person zero-sum differential games have unique saddle point in the sense of Baire’s category.

Existence of value functions of differential games with incomplete information in partially order spaces

In this paper, we consider a two-person zero-sum differential game with incomplete information on the initial state, namely, the first player has a private information while the second player knows only a probability distribution on the initial state, and obtain the existence of its value functions. The main novelty lies in the method for which we apply a generalized weak contraction for dynamic systems and terminal cost functions instead of the Lipschitz contractive condition on a partially ordered metric space.

Differential Games for Fractional-Order Systems: Hamilton–Jacobi–Bellman–Isaacs Equation and Optimal Feedback Strategies

The paper deals with a two-person zero-sum differential game for a dynamical system described by differential equations with the Caputo fractional derivatives of an order α∈(0,1) and a Bolza-type cost functional. A relationship between the differential game and the Cauchy problem for the corresponding Hamilton–Jacobi–Bellman–Isaacs equation with fractional coinvariant derivatives of the order α and the natural boundary condition is established. An emphasis is given to construction of optimal positional (feedback) strategies of the players. First, a smooth case is studied when the considered Cauchy problem is assumed to have a sufficiently smooth solution. After that, to cope with a general non-smooth case, a generalized minimax solution of this problem is involved.

Differential Games with Incomplete Information and with Signal Revealing: The Symmetric Case

In this paper, we investigate the existence of value for a two-person zero-sum differential game with symmetric incomplete information and with signal revealing. Before the game begins, the initial state of the dynamic is chosen randomly among a finite number of points in $$\mathbb {R}^n$$ , while both players have only a probabilistic knowledge of the chosen initial state. During the game, if the system reaches a fixed closed target set K, the current state of the system at the hitting time is revealed to both players. We prove in this paper that this game has a value and its value function is the unique bounded continuous viscosity solution of a suitable Hamilton–Jacobi–Isaacs equation.

One kind of linear-quadratic zero-sum stochastic differential game with jumps

This paper is concerned with one kind of two-person zero-sum linear-quadratic stochastic differential game with jumps. A closed-loop formulation for the saddle points of the game is established in a mixed-strategy-law form, for which the associated Riccati equation and Hamiltonian system are solved.

Unique Ergodicity of Deterministic Zero-Sum Differential Games

We study the ergodicity of deterministic two-person zero-sum differential games. This property is defined by the uniform convergence to a constant of either the infinite-horizon discounted value as the discount factor tends to zero, or equivalently, the averaged finite-horizon value as the time goes to infinity. We provide necessary and sufficient conditions for the unique ergodicity of a game. This notion extends the classical one for dynamical systems, namely when ergodicity holds with any (suitable) perturbation of the running payoff function. Our main condition is symmetric between the two players and involve dominions, i.e., subsets of states that one player can make approximately invariant.

A game-theoretic method for resilient control design in industrial multi-agent CPSs with Markovian and coupled dynamics

Multi-agent cyber-physical systems (CPSs) are ubiquitous in various modern critical infrastructures, especially in industrial automation control involved in the petrochemical, metallurgy, textile, and pharmaceutical, which are vulnerable for their hierarchically and horizontally interconnected architecture. In this paper, we investigate a resilient control design problem for industrial multi-agent CPSs of centralised information structure, with Markovian and physically coupled dynamics from a game-theoretic perspective. First, we investigate a worst-case design problem and solve for worst-case disturbance strategy and optimal disturbance attenuation level parameter through an equivalent two-person zero-sum differential game in both finite- and infinite-horizon. Second, we study a corresponding N-person worst-case nonzero-sum differential game and derive state-feedback Nash equilibrium solution of robustness property as resilient control strategy for each agent. The experimental case study is given to demonstrate that using resilient control strategy, each agent could maintain its physical plant operationally normal under cyber attack, malicious disturbance input and physical state interdependency.

Extremal Shift to Accompanying Points in a Positional Differential Game for a Fractional-Order System

A two-person zero-sum differential game is considered. The motion of the dynamic system is described by an ordinary differential equation with a Caputo fractional derivative of order α ∈ (0, 1). The quality index consists of two terms: the first depends on the motion of the system realized by the terminal time and the second includes an integral estimate of the realizations of the players’ controls. The positional approach is applied to formalize the game in the “strategy–counterstrategy” and “counterstrategy–strategy” classes as well as in the “strategy–strategy” classes under the additional saddle point condition in the small game. In each case, the existence of the value and of the saddle point of the game is proved. The proofs are based on an appropriate modification of the method of extremal shift to accompanying points, which takes into account the specific properties of fractional-order systems.

On Hamilton-Jacobi-Bellman-Isaacs Equation for Time-Delay Systems

The paper deals with a two-person zero-sum differential game for a dynamical system which motion is described by a delay differential equation under an initial condition defined by a piecewise continuous function. For the value functional of this game, we derive the Hamilton-Jacobi type equation with coinvariant derivatives. It is proved that, if the solution of this equation satisfies certain smoothness conditions, then it coincides with the value functional. On the other hand, it is proved that, at the points of coinvariant differentiability, the value functional satisfies the derived Hamilton-Jacobi equation. Therefore, this equation can be called the Hamilton-Jacobi-Bellman-Isaacs equation for time-delay systems.

On Hamilton-Jacobi equations for neutral-type differential games

We consider a two-person zero-sum differential game in which a motion of the dynamical system is described by neutral-type functional-differential equations in Hale’s form and the quality index estimates a motion history realized up to the terminal instant of time and includes integral estimations of control realizations of the players. The formalization of the game in the class of pure positional strategies is given, the corresponding notions of the value functional and optimal control strategies of the players are defined. For the value functional, we derive a Hamilton-Jacobi type equation with coinvariant derivatives. It is proved that, if a solution of this equation satisfies certain smoothness conditions, then it coincides with the value functional. On the other hand, it is proved that, at the points of coinvariant differentiability, the value functional satisfies the derived Hamilton-Jacobi equation. Therefore, this equation can be called the Hamilton-Jacobi-Bellman-Isaacs equation for neutral-type systems.

Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon

This paper is concerned with a linear quadratic stochastic two-person zero-sum differential game with constant coefficients in an infinite time horizon. Open-loop and closed-loop saddle points are introduced. The existence of closed-loop saddle points is characterized by the solvability of an algebraic Riccati equation with a certain stabilizing condition. A crucial result makes our approach work is the unique solvability of a class of linear backward stochastic differential equations in an infinite horizon.

Linear Quadratic Stochastic Differential Games: Open-Loop and Closed-Loop Saddle Points

In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. The existence of an open-loop saddle point is characterized by the existence of an adapted solution to a linear forward-backward stochastic differential equation with constraints, together with a convexity-concavity condition, and the existence of a closed-loop saddle point is characterized by the existence of a regular solution to a Riccati differential equation. It turns out that there is a significant difference between open-loop and closed-loop saddle points. Also, it is found that there is an essential feature that prevents a linear quadratic optimal control problem from being a special case of linear quadratic two-person zero-sum differential games.

Hamilton–Jacobi equations and two-person zero-sum differential games with unbounded controls

A two-person zero-sum differential game with unbounded controls is considered. Under proper coercivity conditions, the upper and lower value functions are characterized as the unique viscosity solutions to the corresponding upper and lower Hamilton–Jacobi–Isaacs equations, respectively. Consequently, when the Isaacs’ condition is satisfied, the upper and lower value functions coincide, leading to the existence of the value function of the differential game. Due to the unboundedness of the controls, the corresponding upper and lower Hamiltonians grow super linearly in the gradient of the upper and lower value functions, respectively. A uniqueness theorem of viscosity solution to Hamilton–Jacobi equations involving such kind of Hamiltonian is proved, without relying on the convexity/concavity of the Hamiltonian. Also, it is shown that the assumed coercivity conditions guaranteeing the finiteness of the upper and lower value functions are sharp in some sense.

An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games

In this paper, a new iterative adaptive dynamic programming (ADP) method is proposed to solve a class of continuous-time nonlinear two-person zero-sum differential games. The idea is to use the ADP technique to obtain the optimal control pair iteratively which makes the performance index function reach the saddle point of the zero-sum differential games. If the saddle point does not exist, the mixed optimal control pair is obtained to make the performance index function reach the mixed optimum. Stability analysis of the nonlinear systems is presented and the convergence property of the performance index function is also proved. Two simulation examples are given to illustrate the performance of the proposed method.

Subsolutions of an Isaacs Equation and Efficient Schemes for Importance Sampling

It was established in Dupuis and Wang [Dupuis, P., H. Wang. 2004. Importance sampling, large deviations, and differential games. Stoch. Stoch. Rep. 76 481–508, Dupuis, P., H. Wang. 2005. Dynamic importance sampling for uniformly recurrent Markov chains. Ann. Appl. Probab. 15 1–38] that importance sampling algorithms for estimating rare-event probabilities are intimately connected with two-person zero-sum differential games and the associated Isaacs equation. This game interpretation shows that dynamic or state-dependent schemes are needed in order to attain asymptotic optimality in a general setting. The purpose of the present paper is to show that classical subsolutions of the Isaacs equation can be used as a basic and flexible tool for the construction and analysis of efficient dynamic importance sampling schemes. There are two main contributions. The first is a basic theoretical result characterizing the asymptotic performance of importance sampling estimators based on subsolutions. The second is an explicit method for constructing classical subsolutions as a mollification of piecewise affine functions. Numerical examples are included for illustration and to demonstrate that simple, nearly asymptotically optimal importance sampling schemes can be obtained for a variety of problems via the subsolution approach.

A Variational Formula for Stochastic Controls and Some Applications

For a controlled stochastic differential equation with a Bolza type performance functional, a variational formula for the functional in a given control process direction is derived, by means of backward stochastic differential equations. As applications, some Pontryagin type maximum principles are established for optimal controls of control problems, for saddle points of open-loop two-person zero-sum differential games, and for Nash equilibria of N -person nonzero-sum differential games. The results presented in this paper generalizes/simplifies the relevant ones found in [12] [17]. In addition, a sufficient existence condition of Nash equilibria is proved for nonzero-sum games.

Differential games with continuous, switching and impulse controls

A two-person zero-sum differential game of infinite duration with discounted payoff involving hybrid controls is studied. The minimizing player is allowed to take continuous, switching, and impulse controls whereas the maximizing player is allowed to take continuous and switching controls. By taking strategies in the sense of Elliott–Kalton, we prove the existence of value and characterize it as the unique viscosity solution of the associated system of quasi-variational inequalities.

A Differential Game with Graph Constrained Switching Strategies

A two-person zero-sum differential game of finite horizon with graphconstrained control strategies is adressed where both players are constrained to use piecewise constant controls, and the control switches are selected from a finite control graph. The control graph is a directed graph where the vertices define pairs of control values for both players, and the edges define the allowable control switches. A positive switching cost is associated with each edge, being the payoff function defined by the sum of the switching costs associated with each player. Issues concerning both the existence of value of the game and optimality conditions are addressed. in terms of value functions. Optimal strategies are derived in terms of a value functions which solve a coupled system of quasi-variational inequalities.